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ABSTRACT 



This paper describes the electronic ACT-R Bookshelf, a 
system which supports learning ACT-R, a well-known theory in the field of 
cognitive psychology, over the World Wide Web. ACT-R Bookshelf is a 
collection of electronic books on various aspects of ACT-R. The primary role 
of ACT-R Bookshelf is to serve as a 24 -hour information resource for Carnegie 
Mellon University (Pennsylvania) on-campus students taking a specific course; 
however, the system can be used as a resource for distance learning of ACT-R, 
as well as an information resource for the international community of 
cognitive scientists and ACT-R researchers. The books on ACT-R Bookshelf are 
authored and served on the Web with the InterBook system. This paper 
discusses concept-based knowledge representation behind adaptive electronic 
textbooks on the bookshelf, describes the main functionality of the system, 
provides some evaluation data, and speculates about prospects for ACT-R 
Bookshelf as an integrated resource on the Web. Two figures present a section 
of ACT-R course material, one with a quiz and the other with a glossary 
window. ( Author /AEF) 
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Abstract: This paper presents an electronic ACT-R bookshelf, a system which supportsleaming 
ACT-R, a well-known theory in the field of cognitive psychologyover the WWW. ACT-R bookshelf 
is a collection of electronic books onvarious aspects of ACT-R. The books on ACT-R bookshelf are 
authored and served on the WWW with InterBook systemwhich makes most of the books on ACT-R 
shelf adaptive. This paper describes main functionality of the Bookshelf, provides some 
evaluationdata, and speculates about possible extensions of Bookshelf- likesystems. 
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1 Introduction 

This paper presents an electronic ACT-R bookshelf, a system which supportsleaming ACT-R, a well-known 
theory in the field of cognitive psychology[Anderson 1993; Anderson & Lebiere 1998], over the WWW. ACT- 
R bookshelf is a collection of electronic books onvarious aspects of ACT-R. The primary role of ACT-R 
bookshelf is to serveas an 24-hours available information resource for Carnegie Mellon University on- 
campusstudents taking course "Production System Models of Thought". However ourintention was to build a 
system which can be used as a resource fordistance learning of ACT-R, as well as an information resource for 
the international community of cognitivescientists and ACT-R researchers. 

A hierarchically structuredelectronic book (EB) is one of the most popular metaphors for representingon- 
line course material. Virtually any kind of course material can be represented as an electronic book. Currently, 
thebookshelf contains an ACT-R tutorial, ACT-R 4.0 User Manual, a Manual forACT-R Lisp-based 
environment, and an ACT-R Glossary. These books are quitedifferent. The ACT-R tutorial is a kind of 
electronic textbook. It provides a step-by-step introduction intoACT-R theory and ACT-R modeling. The 
presentation is linearly sequenced andaugmented with multiple examples, quizzes and assignments. ACT-R 4.0 
UserManual is an encyclopedia-like book for reference access. 

The books on ACT-R bookshelf are authored and served on the WWW withlnterBook system 
[Brusilovsky, Eklund & Schwarz 1998]. Unlike most existing electronic textbooks which are not more than 
astatic electronic copy of a regular textbook: chapter by chapter, page bypage, picture by picture) most of the 
books on ACT-R shelf are adaptive .Adaptivity is especially important foreducational programs on WWW 
which are expected to be used by very differentclasses of students without assistance of a real teacher (who 
usually can provide adaptivity in a normalclassroom). An adaptive nature of ACT-R bookshelf is based on a 
specificconcept-based approach suggested originally in [Brusilovsky 1995] and further elaborated by the ELM 
research group in the process ofdevelopment an adaptive Web-based LISP textbook ELM-ART 
[Brusilovsky, Schwarz & Weber 1996]. InterBook is the first authoring system whichimplements this approach. 
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This paper uncovers concept-based knowledge representation behind adaptive electronic textbooks on the 
Bookshelf, describes mainfunctionality of the system, provides some evaluation data, and speculatesabout 
possible extensions of Bookshelf-like systems. 
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1.1.2 Pr eduction Rules / in ACT-R + 

A production s rule is a statement of a particular 
contingency that controls behavior. Examples might be 

IF the goal-* is to classify a person 
and he is unmarried 
THEN classify him as a bachelor 



IF the goal is to add two digits dl and d2 in a c 
and dl+ d2 = d3 

THEN set as a subgoal-* to write d3 in the column 

The condition s of a production rule (the IF part) consists 
of a specification of a goal and a number of chunks ■> 
while the action s of a production rule (the THEN part) 
basically involves the creation or modifications of some 
chunks. The above is an informal English specification of 
production rules. You will learn the syntax for their 
precise specification within the ACT-R system. 



A production rule specifies an | 
a | is met. 



to be taken when 



Continue 1 Teach me 



HE 
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knowled ge 
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action 
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| Sugg e sted Page : 1 . \ .1 Declarative Units in ACT-R 






Figure 1: A section of ACT-R course material with a quiz as it is presented byNetscape 
browser. The Concept bar (right) shows outcome and backgroundconcepts for the section. 
The Navigation center (top) let the user move inone click to any section on the same or 
upper level. Colored balls (up) and checkmarks (right, on the concept bar)provide adaptive 
annotation. The button "Teach me" provides directguidance. 



2. Knowledge Representation and Content Structuring for Adaptive EB 



The key to adaptivity in an adaptive textbook is knowledge about its domain(i.e., ACT-R) represented in the 
form of domain model and knowledge aboutindividual students represented in the form of individual student 
models.The domain model serves as a basis for structuring the content of an adaptive EB. We distinguish 
twocontent parts in each adaptive EB: a glossary and a textbook . This section provides some minimal 
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information about knowledgerepresentation and content structuring. Some more information could befound in 
[Brusilovsky, Eklund & Schwarz 1998]. 

ACT-R bookshelf uses the simplestform of domain model: a set of domain concepts. By concepts we mean 
elementary pieces ofknowledge for the given domain identified by a domain expert. The domainmodel provides 
a structure for an overlay student model which is arepresentationof the student’s knowledge of the subject. For 
each domain model concept, an individual student’s knowledge model stores some value which is anestimation 
of the student knowledge level of this concept. For ACT-R domainwe distinguish for states of student 
knowledge of any concept: "unknown”, ’’known” (learning started), "learned” and "well-learned”. 

An electronic textbook is a regular book represented in hypermediaform. In InterBook, each EB is 
hierarchically structured into units ofdifferent level: chapters, sections, and subsections. To make EB "more 
intelligent" and toconnect it to the glossary, we have to let the system know what each unitof the textbook is 
about. It is done by indexing of textbook units withdomain model concepts. Several books on the same subject 
form a bookshelf. All books from the same bookshelf areindexed with the same set of domain model concepts. 
Each terminal unit hasan attached list of related concepts (we call this list spectrum of theunit). For each 
involved concept, the spectrum represents the name and the role of the concept in the unit(each concept can be 
either a outcome concept or a prerequisite concept).The system has an option to show all outcome and 
background concepts forthe current section on a page borderto the right of the section content (Figure 1) 
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Netscape: Glossary and Concepts 
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Productions are condition ->- action -» rules vhich specify 
vhat to do in a situation. 

This concept is introduced on these pages: 
ft 1.1.2 Production Rules in ACT-R 
ft 1. 1.3 Production Rule Format 
^ Section 1.5: Creatin g Declarative Structure 

^ Production 

Knowledge about this concept is required for: 

^ Section 1.6: Writin g Productions 
^ Section 2.1: English Rules 

ft S&pctifirnox-sfomic elements 
ft Run Argwmnts 
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Figure 2: A glossary window showing a "page" for production . In addition to providing a 
description of a concept, each glossary entryprovides links to all book sections which 
introduce or require the concept. Colored and checked balls to the left of the links and the 
font typeof the link text provide adaptive annotation. 




The Glossary is, in fact, a visualized domain network. Each node of the domain networkis represented by a 
node of the hyperspace, while the links between domainnetwork nodes constitute main paths between 
hyperspace nodes. The linksbetween domain model concepts constitute navigation paths between glossary 
entries. Thus, the structure of theglossary resembles the pedagogic structure of the domain knowledge. In 
addition to providing a description of a concept,each glossary entry provides links to all book sections which 
introduce orrequire the concept (Figure 2). This means that the glossary integratestraditional features of an 
indexand a glossary. 



3. Functionality 

Domain model-based indexing is a relatively simple but powerful mechanism, because it provides the system 
with knowledge about the content of itspages: the system knows which concepts are presented on each page and 
which concepts have to be learned before starting to learn each page. It opensthe path for several adaptation 
techniques presented in this sub section. 



3.1. Advanced Navigation 

The knowledge about the domain and about the textbook content is used toserve a well-structured hyperspace. 
As any well-designed EB, the systemsupports sequential and hierarchical links for navigation: 

• Each page has back and continue, links which let the user go though thematerial by a linear way 

• The system generates the table of content where all entriesare clickable links to respective sections 

• Each page provides navigation center on the top for one-click navigation to any section onthe same or upper 
level and for understanding "where I am" in thehyperspace. 

In addition the system uses knowledge aboutthe concepts behind the pages to generate other types of links 
between theglossary and the textbook.: 

• The conceptbar provides links from each textbook page to corresponding glossary pagesfor each involved 
concept (Figure 1) 

• The system identifies concept names in the text and turns theminto hyperlinks to the corresponding glossary 
pages (Figure 1) 

• From each glossary page describing a concept the systemprovides links to all textbook units which can be 
used to learn thisconcept (Figure 2). 

All these links are not stored in an external format but generatedon-the-fly by a special module which takes into 
account the student’scurrent state of knowledge represented by the student model. This approachis not only 
reducing page design time but also provides room for adaptation. In particular, our approach supports two 
adaptation techniques: adaptive navigationsupport and prerequisite-based help. 



3.2. Student Modeling, Adaptive Navigation Support, and Adaptive Guidance 

The InterBook approach provides many more opportunities for browsing thecourse materials than traditional 
on-line textbooks. The negative side ofthis is that there is a higher risk for the student to get lost in thiscomplex 
hyperspace. To support the student navigating through the course, the system uses adaptive annotationand direct 
guidance technologies. Adaptive annotation means that the systemuses visual cues (icons, fonts, colors) to show 
the type and theeducational state of each link. Direct guidance means that the system can suggest to the student 
the next part of thematerial to be learned. 

The key to all adaptivefunctionality of ACT-R bookshelf is student modeling. The system maintainsan up- 
to-date model of individual student knowledge on the subject. The student modeling mechanism accepts two 
kinds of evidencesof student knowledge of a concept: 

• a student havevisited a page which presents some information about a concept (i.e., thepage has this concept 
among outcome concepts) 

• a student answer correctly toa test which checks the knowledge of this concept 

The latter evidence is stronger, so no"well-leamed" grade can be given to a concept unless the student 
confirmshis or her knowledge by answering a test. 
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Using the student model, it is possible to distinguish several educational states for each unit ofEB: the 
content of a unit can be known to the student (all outcome conceptshave been already learned), ready to be 
learned, or not ready to be leamed(the latter example means thatsome prerequisite knowledge is not yet 
learned). The icon and the font ofeach link presented to the student are computed dynamically from 
theindividual student model. They always inform the student about the type andthe educational state of the unit 
behind the link. In InterBook, red means not ready to be learned, green meansready and recommended, and 
white means no new information. A checkmark isadded for already visited units (Figures 1 and 2). The same 
mechanism canbe used to distinguish and show several levels of students knowledge of the concepts shown on 
the conceptbar. In InterBook, no annotation means ''unknown", a small checkmark means M known" (learning 
started), a medium checkmark means "learned" and a bigcheckmark means "well-learned" (Figure 1). For many 
students, adaptive guidance provides enough support to makea navigation decision. Those who hesitate to make 
a choice could push thebutton "Teach me” and the system will apply several heuristics to select the most 
suitablenode among those ready to be learned. 



3.3. Prerequisite-BasedHelp 

The system knowledge about the course material comprises knowledge aboutwhat the prerequisite concepts are 
for any unit of the textbook. Often, when students have problems with understanding some explanation or 
example or solving a problem, the reason is that someprerequisite material is not understood well. In that case 
they can requestprerequi site-based help (using a special button) and, as an answer to help request, the 
systemgenerate a list of links to all sections which present some informationabout background concepts of the 
current section. This list is adaptively sorted according to the student'sknowledge represented in the student 
model: more "helpful" sections arelisted first. Here "helpful" means how informative the section is to learn 
about the backgroundconcepts. For example, the section which presents information about anunknown 
background concept is more informative than a section presentinginformation about a known concept. 
Thesection which presents information about two unknown background concepts ismore informative than a 
section presenting information about one concept. 



4. Evaluation 

By the Summer 1998 three groups of students (one group of 10 to 20 studentsper semester) learning ACT-R 
have been using the ACT-R Bookshelf as a leamingresource. Most of the students were undergraduate or 
graduate Psychologystudents. At the first lecture of the course each group of students has 30to 45 minutes 
introduction to ACT-R Bookshelf and its functionality. The Bookshelf was a primary source of 
leaminginformation in addition to regular lectures. No printed handouts wereprovided. 

It was our intention to evaluatehow useful are the advanced features of the Bookshelf for Psychology 
students. Our standard evaluation technology in InterBook is comparingperformance of InterBook users with a 
performance of a control group whichuses a system with advanced functionality disabled [Brusilovsky & 
Eklundl998]. Unfortunately, the number of students taking the ACT-R course is toosmall to run a control 
group. Instead, we have used a more subjectivee valuation technology - a specially developed questionnaire. The 
goal ofthe questionnaire (41 questions in total)was to check whether the users understand the advanced 
functionality of the system (13 questions), whether they find it useful (24 questions), and what kind of 
improvementsthey could suggest (4 questions). We have run the questionnaire with one ofthe groups and 
collected 11 correctly filled forms. Due to the lack of space we could report only some mostinteresting results 
here. 

Our major concern was that Psychology students will not be able tounderstand the advanced functionality 
of the Bookshelf even after a 30-45minutes introduction. To check it we have developed 13 special questions to 
test theirunderstanding. The results show that adaptive navigation support, the most" intelligent" functionality of 
the Bookshelf, is quite clear to thestudents. 11 or 11 students understand the role of the "green ball" and 10 of 
11 understand the role of "red ball"and "checked ball". To compare with , less than half of students were ableto 
understand the functionality of "Search" button, which provides standardsearch functionality (a featureof any 
advanced Web site). At the same time, the students have notconsidered adaptive navigation support as a very 
useful feature. Variouscomponents of it were rated 2.5 to 3.1 on a scale where 1 means totallyuseless and 6 
means very useful. Here we should agree with the students. Adaptive navigation support is not very usefulfor 
well-organized sequentially read textbooks like the ACT-R textbook(here the best strategy is just read all units 
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one after another). Adaptivenavigation support was designed primarily for the users who need to learn selected 
parts of the ACT-Rmaterial and need to use "backward learning" to meet all prerequisites. 



5. Prospects for ACT-RBookshelf as a Integrated Resource on the WWW 

A concept-based indexing of EB on ACT-R bookshelf provide a uniqueopportunity of concept-based linking of 
various kinds of ACT-R relatedinformation. As soon as the ACT-R domain model (a consistent 
andcomprehensive set of ACT-R concepts) is created andall book on the shelf are indexed with ACT-R 
concepts, these books are invisibly linked through the glossary. Eachpage of any book which is indexed with a 
concept will have a link to aglossary entry for this concept. In turn, a glossary entry for a conceptwill include 
the links to all pages in all books on the shelf which are indexed with this concept. Currently, thebookshelf 
contains primary the tutorial and manual-like books. However, weconsider several useful extensions of the 
bookshelf. 

• ACT-R addressbook may list researchers working in ACT-R area providing at least annotated links to their 
home pages. A page inan address book may be indexed with concepts showing area of interest ofthis 
particular researcher within ACT-R domain. 

• An annotated collection of ACT-R research papers may provide a number ofexisting papers on-line. Each 
page in this book may provide an abstract anda URL for one particular paper. This page may be indexed 
with the conceptsdiscussed in this paper as well as with a set of prerequisite concepts required to understand 
it. Somepapers may be represented on a bookshelf as separate books. In this caseeach section of the paper 
may be indexed. 

• The content of main ACT-R book may be indexed section by section and placedon the WWW as a virtual 
copy of the book. When for the copyright reasonsthe content could not be placed on the WWW, the page 
range in the actualbook may be provided enabling auser easy fine a require place in the book. 

There are multiple outcomes of having allACT-R resources on a single bookshelf interconnected by a global 
glossary as presented above. Wherever the userstarts, he or she can see the concepts behind the pages and see a 
glossaryentry for any of these concepts. A glossary entry for a concept willprovide links to all relevant 
information which exists on the bookshelf. For a introductory-level concept, the userwill be able to see book, 
tutorial or manual pages explaining this concept.For more advanced concepts, the user will also see a list of 
papers dealingwith this concept and a listof people investigating this concept deeply. We hope that such 
anintegrated bookshelf accessible worldwide will seriously contribute for thepromotion of ACT-R learning and 
research worldwide. 
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